Graph Convolutional Neural Networks Research Articles

Capturing Local Temperature Evolution during Additive Manufacturing through Fourier Neural Operators

Abstract High-fidelity, data-driven models that can quickly simulate thermal behavior during additive manufacturing (AM) are crucial for improving the performance of AM technologies in multiple areas, such as part design, process planning, monitoring, and control. However, the complexities of part geometries make it challenging for current models to maintain high accuracy across a wide range of geometries. Additionally, many models report a low mean square error (MSE) across the entire domain of a part. However, in each time step, most areas of the domain do not experience significant changes in temperature, except for the regions near recent depositions. Therefore, the MSE-based fidelity measurement of the models may be overestimated. This paper presents a data-driven model that uses the Fourier Neural Operator to capture the local temperature evolution during the AM process. Beside MSE, the model is also evaluated using the R2 metric, which places great weight on the regions where the temperature changes significantly than MSE. The model was trained and tested on numerical simulations based on the Discontinuous Galerkin Finite Element Method for the Direct Energy Deposition AM process. The results shows that the model maintains 0.983-0.999 R2 over geometries not included in the training data, which is higher than Convolutional Neural Networks and Graph Convolutional Neural Networks we implemented, the two widely used architectures in data-driven predictive modeling.

StellarPath: Hierarchical-vertical multi-omics classifier synergizes stable markers and interpretable similarity networks for patient profiling.

The Patient Similarity Network paradigm implies modeling the similarity between patients based on specific data. The similarity can summarize patients' relationships from high-dimensional data, such as biological omics. The end PSN can undergo un/supervised learning tasks while being strongly interpretable, tailored for precision medicine, and ready to be analyzed with graph-theory methods. However, these benefits are not guaranteed and depend on the granularity of the summarized data, the clarity of the similarity measure, the complexity of the network's topology, and the implemented methods for analysis. To date, no patient classifier fully leverages the paradigm's inherent benefits. PSNs remain complex, unexploited, and meaningless. We present StellarPath, a hierarchical-vertical patient classifier that leverages pathway analysis and patient similarity concepts to find meaningful features for both classes and individuals. StellarPath processes omics data, hierarchically integrates them into pathways, and uses a novel similarity to measure how patients' pathway activity is alike. It selects biologically relevant molecules, pathways, and networks, considering molecule stability and topology. A graph convolutional neural network then predicts unknown patients based on known cases. StellarPath excels in classification performances and computational resources across sixteen datasets. It demonstrates proficiency in inferring the class of new patients described in external independent studies, following its initial training and testing phases on a local dataset. It advances the PSN paradigm and provides new markers, insights, and tools for in-depth patient profiling.

Novel GCN Model Using Dense Connection and Attention Mechanism for Text Classification

Convolutional Neural Network (CNN) or Recurrent Neural Network (RNN) based text classification algorithms currently in use can successfully extract local textual features but disregard global data. Due to its ability to understand complex text structures and maintain global information, Graph Neural Network (GNN) has demonstrated considerable promise in text classification. However, most of the GNN text classification models in use presently are typically shallow, unable to capture long-distance node information and reflect the various scale features of the text (such as words, phrases, etc.). All of which will negatively impact the performance of the final classification. A novel Graph Convolutional Neural Network (GCN) with dense connections and an attention mechanism for text classification is proposed to address these constraints. By increasing the depth of GCN, the densely connected graph convolutional network (DC-GCN) gathers information about distant nodes. The DC-GCN multiplexes the small-scale features of shallow layers and produces different scale features through dense connections. To combine features and determine their relative importance, an attention mechanism is finally added. Experiment results on four benchmark datasets demonstrate that our model’s classification accuracy greatly outpaces that of the conventional deep learning text classification model. Our model performs exceptionally well when compared to other text categorization GCN algorithms.

Transformer Encoder Enhanced by an Adaptive Graph Convolutional Neural Network for Prediction of Aero-Engines’ Remaining Useful Life

Accurate prediction of remaining useful life (RUL) plays a significant role in ensuring the safe flight of aircraft. With the recent rapid development of deep learning, there has been a growing trend towards more precise RUL prediction. However, while many current deep learning methods are capable of extracting spatial features—those along the sensor dimension—through convolutional kernels or fully connected layers, their extraction capacity is often limited due to the small scale of kernels and the high uncertainty associated with linear weights. Graph neural networks (GNNs), emerging as effective approaches for processing graph-structured data, explicitly consider the relationships between sensors. This is akin to imposing a constraint on the training process, thereby allowing the learned results to better approximate real-world situations. In order to address the challenge of GNNs in extracting temporal features, we augment our proposed framework for RUL prediction with a Transformer encoder, resulting in the adaptive graph convolutional transformer encoder (AGCTE). A case study using the C-MAPSS dataset is conducted to validate the effectiveness of our proposed model.

A fast prediction model of blade flutter in turbomachinery based on graph convolutional neural network

With the development of gas turbine towards higher load capacity and improved efficiency, blade flutter has emerged as a significant obstacle in the further development of turbomachinery due to fluid-structure coupling phenomenon. However, the traditional numerical simulation methods lengthen the design cycle and reduce the optimization efficiency due to the expensive time and resource cost. To solve this problem, a fast prediction model of blade flutter based on the graph convolutional neural network is proposed. The blade flutter prediction graph convolutional neural network containing two sub-networks. The flow field prediction network is used to realize prediction from design variables to flow field parameters. The flutter parameter identification network is adopted to realize identification from flow field parameters to 15 flutter parameters. The prediction performance for physical fields, modal forces, aerodynamic damping and aeroelastic stability are discussed. The influence of training size is explored. The results show that the network can not only accurately capture the trends and details of the physical field, but also obtain the accurate modal force curve, so as to calculate the aerodynamic damping parameters and finally judge the aeroelastic stability.

Intelligent recommendation system for College English courses based on graph convolutional networks

With the rapid development of international communication, the number of English courses has shown an explosive growth trend, which has caused a serious problem of information overload, resulting in poor teaching performance of recommended English courses. To solve this problem, this paper proposes a graph convolutional neural network model based on College English course texts, students’ major, English foundation and network structure characteristics. First, by analyzing the relevant data of College English courses and combining with graph neural network, an English course recommendation algorithm model based on the College English learning strategy of proximity comparison is proposed. Then, the College English texts are taken as feature input, and multi-layer graph convolutional neural network is used to process the above graph neural network structure. Attention mechanism is introduced to enhance the representation of graph features in College English skills. Finally, multi-layer attention model is used to process the courses that users have learned, and intelligent course recommendation is made by combining the multi-layer attention modeling of College English skills. The experimental data show that the proposed method achieves the best performance compared with the commonly used College English course recommendation method.

Spectral Graph Convolutional Neural Networks in the Context of Regularization Theory.

Graph convolutional neural networks (GCNN) have been widely used in graph learning and related applications. It has been identified that the filters in the state-of-the-art spectral graph convolutional networks (SGCNs) are essentially low-pass filters that enforce smoothness across the graph and use the functions of graph Laplacian as a tool that injects graph structure into the learning algorithm. There had been research findings that connect the smoothness functional in graphs, graph Laplacian, and regularization operators. We review the existing SGCNs in this context and propose a framework where the state-of-the-art filter designs can be deduced as special cases. We designed new filters that are associated with well-defined low-pass behavior and tested their performance on semisupervised node classification tasks. Their performance was found to be superior to that of the other state-of-the-art techniques. We further investigate the representation capability of low-pass features and make useful observations. In this context, we discuss a few points to further optimize the network, new strategies for designing SGCNs, their challenges, and some latest related developments. Based on our framework, we also deduce the connection between support vector kernels and SGCN filters.

Simplified neural architecture for efficient human motion prediction in human-robot interaction

Human motion prediction is essential for safe and effective human-robot interaction, but modeling the intricate spatio-temporal dynamics inherent in human movement remains challenging. While recent methods have advanced motion prediction accuracy, they rely on complex neural architectures such as Graph Convolutional Networks and Recurrent Neural Networks, demanding extensive hyperparameter tuning. To overcome this limitation, we propose a streamlined multi-stage layer incremental perceptron (MSLP) architecture that achieves competitive results with far fewer parameters, improving efficiency. The MSLP eschews the complexity of prevalent networks and instead takes a stepped refinement approach to predict intricate motions using a lightweight model. This simplified yet effective design enables nuanced learning of spatio-temporal relationships without exhaustive tuning. Specifically, the MSLP incorporates two key components: a multi-channel feature extraction and enhancement block (MFE-block) and a temporal feature extraction module (TFE-block). The MFE-block strengthens the representation of each action node by integrating multi-dimensional action features. The TFE-block then captures the contextual relationships between actions over time. Together, the MFE-block and TFE-block allow the MSLP to model the complex spatial and temporal dynamics of human movement using a streamlined architecture and minimal parameter tuning. When evaluated on established datasets including Human3.6M, CMU, 3DPW, and AMASS, the proposed MSLP method achieves improved accuracy gains of 16.7%-67.0% over existing state-of-the-art techniques. Additionally, the MSLP significantly reduces the number of parameters by 35.7%-99.5% compared to prior architectures. We find that the proposed MSLP significantly improves both long-term prediction and generalization capabilities of the model.

FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently

FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently

Automated generative design and prefabrication of precast buildings using integrated BIM and graph convolutional neural network

Precast construction is a productivity-improving technology in the architectural, engineering, and construction industry that improves construction efficiency by combining factory-based manufacturing and lean assembly. Many international efforts have encouraged the adoption of this approach. This study presents an integrated Building Information Modelling (BIM) with technological automation interoperability to enable generative design and prefabrication for precast buildings. A generic BIM-based graph representation is established to explicitly formulate buildings' spatial and geometric features. Following this, a graph-constrained layout generator is developed, with a generative modelling algorithm and graph convolutional neural network, to extract pairwise spatial-geometric features for generating the optimal precast layout. This is followed by semantic enrichment of BIM data (i.e., Industry Foundation Classes) with precast data schema to facilitate data transformation for prefabrication automation until site delivery. The holistic approach presented in this study empowers pre-construction planning optimisation and fabrication automation in precast construction.

SEGCECO: Subgraph Embedding of Gene expression matrix for prediction of CEll-cell COmmunication.

Recent advances in single-cell RNA sequencing technology have eased analyses of signaling networks of cells. Recently, cell-cell interaction has been studied based on various link prediction approaches on graph-structured data. These approaches have assumptions about the likelihood of node interaction, thus showing high performance for only some specific networks. Subgraph-based methods have solved this problem and outperformed other approaches by extracting local subgraphs from a given network. In this work, we present a novel method, called Subgraph Embedding of Gene expression matrix for prediction of CEll-cell COmmunication (SEGCECO), which uses an attributed graph convolutional neural network to predict cell-cell communication from single-cell RNA-seq data. SEGCECO captures the latent and explicit attributes of undirected, attributed graphs constructed from the gene expression profile of individual cells. High-dimensional and sparse single-cell RNA-seq data make converting the data into a graphical format a daunting task. We successfully overcome this limitation by applying SoptSC, a similarity-based optimization method in which the cell-cell communication network is built using a cell-cell similarity matrix which is learned from gene expression data. We performed experiments on six datasets extracted from the human and mouse pancreas tissue. Our comparative analysis shows that SEGCECO outperforms latent feature-based approaches, and the state-of-the-art method for link prediction, WLNM, with 0.99 ROC and 99% prediction accuracy. The datasets can be found at https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE84133 and the code is publicly available at Github https://github.com/sheenahora/SEGCECO and Code Ocean https://codeocean.com/capsule/8244724/tree.

Flexible drug-target interaction prediction with interactive information extraction and trade-off

Drug-target interaction (DTI) prediction refers to the use of computational methods and models to predict the interaction between drugs and biological targets. DTI can help researchers understand the mechanism of action of drugs, discover new drug targets, and screen drug candidates. Recently, a large number of DTI models integrating deep drug-target interaction features have emerged to make up for the dilemma of incomplete information on shallow drug and target features. However, these models ignore the challenge of overlapping interaction information by simply integrating deep interaction information. This paper proposes a flexible DTI prediction with interactive information extraction and trade-off (FDTIIT) to address the above challenges. The main idea of FDTIIT is to use flexible mutual attention to extract interaction information about drugs and targets, and then limit the dependence between them to avoid redundant information. Specifically, FDTIIT mainly includes three parts: drug and target representation, drug-target interactive information extraction, and drug-target interactive information trade-off. Among them, the drug and target representation module mainly uses the graph convolutional network and convolutional neural network to learn the representation of drugs and targets. Then, the drug-target interactive information extraction module extracts the drug information hidden in the target and the target information hidden in the drug based on mutual attention. To avoid possible information overlap between drug representation and target representation after the fusion of interaction information, FDTIIT designs an interactive information trade-off module. This module limits the dependence between drug and target representation, providing more comprehensive information to support high-performance drug-target interaction prediction. Multiple experiments designed on three publicly available datasets validated FDTIIT’s effectiveness.

Data and physics-driven modeling for fluid flow with a physics-informed graph convolutional neural network

A physics-informed graph convolutional network (PINN-GCN) data-driven model was proposed to solve steady-state fluid flows with various geometries. A GCN guarantees a data-driven model that adapts to irregularly structured or unstructured mesh data. In addition, the mean squared residuals of the governing equations in partial differential forms and boundary conditions are introduced into the loss function of the network model, the Physics-Informed Neural Network (PINN) increases the knowledge of the physical rules of the model, enhances the model accuracy even with sparse training data. With nine training cases containing approximately 2000 spatial point data individually, PINN-GCN could accurately predict the physical flow field over a two-dimensional object of variable size, achieving above 98.6% for mean accuracy. With five training data cases, the model could predict flow over a cylinder with variable position, achieving above 98.2% for mean accuracy. Moreover, the PINN-GCN framework can produce high-resolution predictions, even with low-resolution field training data. Furthermore, the PINN-GCN prediction speed was approximately 1000 times faster than that of the numerical simulations. Thus, the proposed model can rapidly predict high-resolution flow fields using a physics-informed, grid- and geometry-adaptive, data-driven model.

Physics-Informed Graph Neural Networks for Water Distribution Systems

Water distribution systems (WDS) are an integral part of critical infrastructure which is pivotal to urban development. As 70% of the world's population will likely live in urban environments in 2050, efficient simulation and planning tools for WDS play a crucial role in reaching UN's sustainable developmental goal (SDG) 6 - "Clean water and sanitation for all". In this realm, we propose a novel and efficient machine learning emulator, more precisely, a physics-informed deep learning (DL) model, for hydraulic state estimation in WDS. Using a recursive approach, our model only needs a few graph convolutional neural network (GCN) layers and employs an innovative algorithm based on message passing. Unlike conventional machine learning tasks, the model uses hydraulic principles to infer two additional hydraulic state features in the process of reconstructing the available ground truth feature in an unsupervised manner. To the best of our knowledge, this is the first DL approach to emulate the popular hydraulic simulator EPANET, utilizing no additional information. Like most DL models and unlike the hydraulic simulator, our model demonstrates vastly faster emulation times that do not increase drastically with the size of the WDS. Moreover, we achieve high accuracy on the ground truth and very similar results compared to the hydraulic simulator as demonstrated through experiments on five real-world WDS datasets.

Learning to Pivot as a Smart Expert

Linear programming has been practically solved mainly by simplex and interior point methods. Compared with the weakly polynomial complexity obtained by the interior point methods, the existence of strongly polynomial bounds for the length of the pivot path generated by the simplex methods remains a mystery. In this paper, we propose two novel pivot experts that leverage both global and local information of the linear programming instances for the primal simplex method and show their excellent performance numerically. The experts can be regarded as a benchmark to evaluate the performance of classical pivot rules, although they are hard to directly implement. To tackle this challenge, we employ a graph convolutional neural network model, trained via imitation learning, to mimic the behavior of the pivot expert. Our pivot rule, learned empirically, displays a significant advantage over conventional methods in various linear programming problems, as demonstrated through a series of rigorous experiments.

ASWT-SGNN: Adaptive Spectral Wavelet Transform-Based Self-Supervised Graph Neural Network

Graph Comparative Learning (GCL) is a self-supervised method that combines the advantages of Graph Convolutional Networks (GCNs) and comparative learning, making it promising for learning node representations. However, the GCN encoders used in these methods rely on the Fourier transform to learn fixed graph representations, which is inherently limited by the uncertainty principle involving spatial and spectral localization trade-offs. To overcome the inflexibility of existing methods and the computationally expensive eigen-decomposition and dense matrix multiplication, this paper proposes an Adaptive Spectral Wavelet Transform-based Self-Supervised Graph Neural Network (ASWT-SGNN). The proposed method employs spectral adaptive polynomials to approximate the filter function and optimize the wavelet using contrast loss. This design enables the creation of local filters in both spectral and spatial domains, allowing flexible aggregation of neighborhood information at various scales and facilitating controlled transformation between local and global information. Compared to existing methods, the proposed approach reduces computational complexity and addresses the limitation of graph convolutional neural networks, which are constrained by graph size and lack flexible control over the neighborhood aspect. Extensive experiments on eight benchmark datasets demonstrate that ASWT-SGNN accurately approximates the filter function in high-density spectral regions, avoiding costly eigen-decomposition. Furthermore, ASWT-SGNN achieves comparable performance to state-of-the-art models in node classification tasks.

Mixed-integer optimisation of graph neural networks for computer-aided molecular design

ReLU neural networks have been modelled as constraints in mixed integer linear programming (MILP), enabling surrogate-based optimisation in various domains and efficient solution of machine learning certification problems. However, previous works are mostly limited to MLPs. Graph neural networks (GNNs) can learn from non-euclidean data structures such as molecular structures efficiently and are thus highly relevant to computer-aided molecular design (CAMD). We propose a bilinear formulation for ReLU Graph Convolutional Neural Networks and a MILP formulation for ReLU GraphSAGE models. These formulations enable solving optimisation problems with trained GNNs embedded to global optimality. We apply our optimisation approach to an illustrative CAMD case study where the formulations of the trained GNNs are used to design molecules with optimal boiling points.

RWESA-GNNR: Fusing Random Walk Embedding and Sentiment Analysis for Graph Neural Network Recommendation

A graph neural network-based recommendation system treats the relationship between user items as a graph, and achieves deep feature mining by modelling the graph nodes. However, the complexity of the features of graph neural network-based recommendation systems brings poor interpretability and suffers from data sparsity problems. To address the above problems, a graph convolutional neural network recommendation model (RWESA-GNNR) based on random walk embedding combined with sentiment analysis is proposed. Firstly, a random walk-based matrix factorization is designed as the initial embedding. Secondly, the user and item nodes are modelled using a convolutional neural network with an injected attention mechanism. Then, sentiment analysis is performed on the review text, and attention mechanism is introduced to fuse text sentiment features and semantic features. Finally, node features and text features are aggregated to generate recommendation results. The experimental results show that our proposed algorithm outperforms traditional recommendation algorithms and other graph neural network-based recommendation algorithms in terms of recommendation results, with an improvement of about 2.43%-5.75%.

Multi-sequence spatio-temporal feature fusion network for peak-hour passenger flow prediction in urban rail transit

ABSTRACT This research addresses the challenge of predicting URT station passenger flow during peak hour. The Multi-Sequence Spatio-Temporal Feature Fusion Network Model (MSSTFFN) based on trend decomposition is introduced to capture complex spatio-temporal correlations. This model combines seasonal trend decomposition, graph convolutional neural networks, and modified Transformer networks. The MSSTFFN model is evaluated using actual data from Hangzhou City. The results indicate that, in comparison to the baseline model, this model consistently delivers the best prediction results across various datasets as well as prediction tasks. It exhibits exceptional and consistent performance in prediction sub-tasks involving different input and prediction step combinations, highlighting its advanced, robust, and versatile nature. Through micro-comparisons of specific prediction results for different types of stations, the practical application value is verified. Furthermore, through the design of ablation experiments and testing on various datasets, the contribution value of the features and model’s generalization capability are validated.

A Novel Money Laundering Prediction Model Based on a Dynamic Graph Convolutional Neural Network and Long Short-Term Memory

Money laundering is an illicit activity that seeks to conceal the nature and origins of criminal proceeds, posing a substantial threat to the national economy, the political order, and social stability. To scientifically and reasonably predict money laundering risks, this paper focuses on the “layering” stage of the money laundering process in the field of supervised learning for money laundering fraud prediction. A money laundering and fraud prediction model based on deep learning, referred to as MDGC-LSTM, is proposed. The model combines the use of a dynamic graph convolutional network (MDGC) and a long short-term memory (LSTM) network to efficiently identify illegal money laundering activities within financial transactions. MDGC-LSTM constructs dynamic graph snapshots with symmetrical spatiotemporal structures based on transaction information, representing transaction nodes and currency flows as graph nodes and edges, respectively, and effectively captures the relationships between temporal and spatial structures, thus achieving the dynamic prediction of fraudulent transactions. The experimental results demonstrate that compared with traditional algorithms and other deep learning models, MDGC-LSTM achieves significant advantages in comprehensive spatiotemporal feature modeling. Specifically, based on the Elliptic dataset, MDGC-LSTM improves the Macro-F1 score by 0.25 compared to that of the anti-money laundering fraud prediction model currently considered optimal.